For centuries, only the wealthy or privileged classes of society could afford to employ the skilled artisans who labored to produce a fine likeness in painting, sculpture, and drawing. In many cases, portraiture served a purpose greater than the simple creation of an acceptable likeness of reality. In subtle or overt ways, the artist's work would interact with the desires and intentions of the subjects. A second category of artistic license involved improvement on reality. Thus, subjects were rendered in such a way as to minimize their physical imperfections and to present the most attractive possible appearance.
In modern society, portraiture is no longer the exclusive domain of the wealthy and powerful. The advent of photography into all levels of society has rendered creation of portrait images to be an ubiquitous part of many of life's major events. Weddings, graduations, birthdays, arrival of a baby—all of these events, and more—are commonly captured with relatively standardized portrait images in western cultures. While the state of technology enables individual amateurs to capture and even enhance images such as these, there still exists a class of professional photographers that provide creation of higher-quality portrait images. Not surprisingly, the goals of the portraitist remain the same as in bygone centuries—to present the subject in the most pleasing possible way. In essence, the subject wants to be seen as they wish they were, not as they really are.
In response to the desire for people to be seen as they wish they are, and not the way they really are, professional photographers resort to retouching the portrait image to give people their preferred appearance. Retouching involves changing a photo image in some way that was not captured or depicted in the original photographic image. One of the goals of retouching a portrait image is to make a person look better by removing temporary imperfections such as blemishes or dark circles under the eyes or permanent imperfections such as moles or wrinkles, while still maintaining the personality of the individual. Removing facial blemishes, moles and scars, softening lines and wrinkles, decreasing bags under the eyes, whitening teeth and the whites of the eyes are examples of retouching performed to improve or enhance the appearance of an individual in a portrait image.
Before the advent of the digital age, retouching of images were performed on either the negative or printed image by modifying the image using dyes to mask or change imperfections in the portrait image. Now that digital image capture devices are routinely available, the preferred method of retouching is done via digital imaging techniques performed on the captured digital images. Digital methods allow enhancements to be performed that were either extremely hard to do or previously impossible to perform on the analogue image. Image editing software such as Adobe Photoshop® can be used to refine portraits by removing blemishes, straightening noses, balancing eyes and applying digital make-up.
Improvements in computer technology and image processing algorithms are enabling new classes of automated and semi-automated image enhancements. Relating to the subject of portrait images, relevant technological developments include face detection and recognition, facial feature detection and masking, face re-posing, and red-eye detection and correction.
In published PCT Patent Application WO 00/76398 A1, “Skin Imaging Analysis Systems and Methods”, Hillebrand et al. disclose a system that can detect skin defects and calculate a skin severity index. This system is aimed towards the cosmetic and skin care market. The system can also simulate improvements to the defected skin areas that would be realized upon the use of a recommended treatment or product that eliminates or hides the skin defect. The skin defects are detected using color information and standard morphing techniques are used to simulate improvements in the defected skin areas.
In published European Patent Application EP 1 030 267 A1, “Method of Correcting Face Image, Makeup Simulation Method, Makeup Method, Makeup Supporting Device and Foundation Transfer Film”, Utsugi describes a method for preparing an ideal post-makeup face through image processing based on a desirable face or a model face. The technique involves making highlighted areas, and the shapes of the eyebrows, the eyelines, and the lip line closer to that of a desirable face or a model face within a range where modification by makeup is possible.
The drawback of such systems, especially for batch portraiture systems as used, e.g., for weddings, graduations, school and sports pictures, birthdays, arrival of a baby, etc., is the intense interaction required with the customer to input preferences and evaluate results. For example, in Utsugi the makeup customer's presence is required to settle on the model face, e.g., selected from the faces of popular talents, actresses or actors, and on the various adjustments made to reach the model face. Moreover, a skilled operator is required to work with the customer to produce an acceptable result.
Even with the advent of digital imaging, therefore, retouching portraits is a craft unto itself and to this day remains more of an art form than a science. In addition, the process of retouching portrait images is a highly manual and time consuming process performed by skilled operators. It therefore would be advantageous to develop a system that uses automated and semi-automated portrait image enhancement methods to enable the facile retouching of portraits. The present invention solves the above mentioned shortcomings of the current art by providing methods and system for automated enhancement of the appearance of the human subjects in images.